Linear Versus Non-Linear Policy Capturing in a Dynamic Classification Task

Policy capturing is a decision analysis technique normally using linear statistical models to infer the basis of expert judgments. The purpose of the present work was to test if the C4.5 decision tree (DT) algorithm (a non-linear machine learning method) is more effective at capturing individual’s decision policies than the standard linear technique. Human classification behavior was measured in a simulated naval air-defence task to compare decision tree models and linear logistic regression models in terms of their descriptive and predictive accuracy. Results show that C4.5 was superior in terms of goodness-of-fit and cross-validation performance. Decision tree complexity was significantly correlated to individuals’ response times. The classification rules derived from each individual were actually more reliable than their human counterparts – replicating a classic finding in policy capturing. We conclude that C4.5 is a useful policy capturing tool in the context of a complex dynamic task environment.